The Development of a Biographical Inventory for Use in the Prediction of First-year College Success

This study utilized an originally developed biographical inventory, the Behavioral Indicators of Future Performance (BIFP), to establish valid non-cognitive variables predictive of initial college success. The BIFP was administered in 2005 to college freshmen entering their first semester and included a final sample of 393 men and 295 women. Each of 308 variables on the BIFP was assessed for its relationship to college success, defined as a 3.0 Grade Point Average, in both developmental and cross-validation samples. Thirty-seven variables were significantly correlated with college success. When used together, these 37 variables were more highly correlated with first semester GPA than the SAT. Findings can ultimately be used to increase the predictive validity of the college selection process. INTRODUCTION What makes a successful person? Vince Lombardi observed, "The difference between a successful person and others is not a lack of strength, not a lack of knowledge, but rather in a lack of will” (The Official Site of Vince Lombardi, n.d.). He recognized, as do so many of us, that motivation may be the most important factor in determining success. In the field of research, however, motivation and biographical factors conceptually related to success cannot be directly measured and are difficult to manipulate as variables. Therefore, a limited number of studies have successfully linked such factors to success in specific areas. In the educational environment, a student’s motivation, success orientation, and network of support can surpass the importance of writing and reading skills. However, colleges and universities do not currently acknowledge such factors in their selection process, which is intended to choose students most likely to succeed at their school. Most colleges and universities limit their admission decisions to the traditional cognitive predictors of academic success, usually high school grade point average (GPA), scores on standardized tests such as the SAT, and high school rank (Aiken, 1964). The purpose of this study is to identify non-cognitive predictors of college success by researching biographical information, and its relationship to students’ performance in their first semester of college. The findings of this research can ultimately increase the predictive ability and fairness of admission decisions in colleges and universities. Although traditional cognitive measurements can predict college success to some degree, problems remain with each. For example, many colleges use high school GPA as a primary predictor of future academic success. While this may appear logical under the assumption that past academic performance is the best predictor of future academic performance, this assumption is flawed because high school GPAs are not nationally standardized. Students may attend a high school that is more challenging than others, and a qualified student who graduated from a top-ranked high school could be rejected because of a GPA lower than that of other applicants who graduated from a less rigorous high school. An additional problem is that high schools use varying scales to measure GPA, with some schools choosing to assign additional weight to honors and advanced placement courses, and others choosing not to weight these courses or weigh them differently. A problem with the use of high school rank in college selection is that many high schools do not rank, resulting in missing data for such applicants. In addition, because some high schools are more M. CIRILLO and T. SMITH: USING BIOGRAPHICAL DATA TO PREDICT COLLEGE SUCCESS -2competitive than others, it is more difficult for a student to obtain higher rankings at a more competitive high school. As a result, this variable is biased against those students graduating from more competitive high schools. It can be argued that this bias punishes students for challenging themselves (in attending a more competitive high school), a message that those who work in higher education would almost certainly disagree with as a matter of principle. Standardized tests such as the SAT also have many problems as predictor variables. Studies have shown the SAT to be culturally and statistically biased against minorities and other non-traditional students (those not from upper to middle socio-economic backgrounds) (Freedle, 2002). The test also accounts for only a small portion, 15 percent, of what it is intended to predict, first year college GPA (Lemann, 1999). Taken together, the traditional cognitive variables explain only 25 percent of variation in first-year success. To continue to use these methods in college selection when we know of their shortcomings encourages neglect and denial of the motivational and biographical factors associated with success. One possible tool that may fill the void left by current measures is the use of biographical data (biodata) in the selection process. It is believed that one can use past performance to predict future performance using biodata, past biographical accounts of experiences and events (Gunter, Furnham, and Drakely, 1993). Biographical inventories are surveys that assess an individual’s biographical history, as well as considering that person’s attitudes, expectations, and preferences. When used in selection, biographical inventories have been shown to be less biased against minority groups than cognitive ability tests (Beasely and Sease, 1974). Using this type of survey or questionnaire to evaluate college applicants would encourage a more fair and accurate selection process. A number of studies have shown that several variables other than academic characteristics are associated with success in college. Larose, Robertson, Roy, and Legault (1998) argue that the frequency and quality of the interactions between students and their faculty are related to higher levels of college success. Other variables related to college success are the ability to adapt to the college environment, personal motivation, and students’ relationships with their peers (Larose et al., 1998). In addition to these findings, Aiken (1964) suggests that students who withdraw from classes are less motivated to achieve than students who do not withdraw. A study conducted by Naumann, Bendalos, and Gutkin (2003) at Midwestern University implies that first-generation college students rely more heavily on motivational factors in order to succeed than do second-generation students. These results were not surprising given “first-generation students did not typically have the same sources of support throughout their educational careers as did secondgeneration students” (Naumann, Bendalos, and Gutkin, 2003, p. 8). The study conducted by Larose et al. (1998) found that behavioral scales, based on nonintellectual learning dispositions, were actually better predictors of college success (measured by GPA) than were the SAT and high school rank. Additionally, within ethnic minority populations, correlations between biographical inventories and GPA have been higher than the correlations obtained by the SAT (Beasley and Sease, 1974). It is clear that non-cognitive variables play a role in college success, and these variables can be measured with biographical inventories. Thus, in order to discover several different non-cognitive predictors of college success and build on past research, a biographical inventory was developed to assess college freshmen at a four-year liberal arts college, The College of New Jersey (TCNJ). The biographical inventory was generated by Kirnan, Tsui, and Stephens (2006), and is called the Behavioral Indicators of Future Performance (BIFP). The BIFP was developed using multiple sources of information: an intensive examination of past studies and other relevant literature was performed; interviews were conducted with 15 college students (4 juniors, 4 seniors, 6 sophomores, and one recent graduate) as well as 13 subject matter experts who averaged 16 years of working in higher education (Kirnan, Tsui, and Stephens, 2006); and notes were recorded for each interview and later analyzed for common themes. Themes extracted included time management, personal characteristics, distractions, maturity, level of support, realistic expectations, goal setting, responsibility, and writing ability. The research team1 then created an eight-page draft of the BIFP survey composed of 45 items, which were created based on the extracted themes and grouped into eight variable categories: (1) TCNJ JOURNAL OF STUDENT SCHOLARSHIP VOLUME X APRIL, 2008 -3Expectations, the extent to which the student has a realistic view of what the first year of college will demand; (2) Academic Preparedness, the extent to which the student has acquired academic skills needed for college; (3) Technological Savvy, technological skills and knowledge of the student; (4) Independence, the ability for the student to be self-reliant; (5) Distractions, factors unrelated to college work that interfere with academic success; (6) Ease of Adjustment, extent to which the student can adapt to a new environment and is open to new experiences, also including the available social support of the individual; (7) Personal Habits, the extent to which the student has displayed a good work ethic in school and/or in employment; and (8) Success Orientation, the extent to which the student has a desire to succeed (Kirnan, Tsui, and Stephens, 2006). Once the draft of the BIFP was completed, it was assessed by two focus groups consisting of 12 freshmen at TCNJ, 4 male, 8 female; 7 Caucasian, 5 of an ethnic minority. They completed the BIFP and wrote down any comments or questions they had. Two facilitators then led a discussion with each group in which participants were asked for feedback on items. Based on the focus group proceedings, changes and revisions were made when necessary (Kirnan et al., 2006). The purpose of the current research is to analyze each item of the BIFP for its ability to predict college success using a correlational method. Each item will be retained, reworded, or eliminated from the BIFP depending on its ability to predict success. A total biodata score will be derived from those items that demonstrate an acceptable correlation with success. Specifically, researchers hypothesized that the BIFP biodata score would correlate with early college success. In addition, they assumed that the BIFP score would provide incremental validity when combined with traditional cognitive predictors such as SAT and high school rank. Finally, the team predicted that these findings would cross-validate to a hold-out sample, showing some shrinkage but still yielding a correlation between the BIFP score and success, and a unique contribution in prediction relative to traditional measures.